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LONG-TERM  GOALS 

In  this  final  year  of  a  3  year  effort  by  the  Naval  Research  Laboratory  Code  7544  to  enable  Navy 
aerosol  forecasting  to  take  full  advantage  of  available  data  feeds  through  the  investigation  of 
fundamental  atmospheric  composition  observability,  we  find  that  the  next  generation  of  data  feeds  and 
associated  technology  present  a  number  of  challenges  and  opportunities  which  require  attention.  These 
include:  1)  The  transition  from  NASA  EOS  sensors  to  the  next  generation  of  diversified  operational 
and  near  real  time  data  sources;  2)  The  move  to  a  constellation  approach  for  global  atmospheric 
composition  observing,  3)  The  expected  near  real  time  availability  of  US  and  European  lidar  data;  4) 
The  enhanced  availability  of  surface  and  aircraft  observations;  and  5)  Increased  aerosol  model 
demands  for  such  applications  as  joint  surface-atmosphere  retrievals,  directed  energy  (DE),  and 
intelligence,  surveillance,  and  reconnaissance  (ISR).  This  increase  in  the  number  of  potential  data 
sources,  coupled  with  further  efficacy  demands,  creates  an  imperative  need  to  understand  the  nature  of 
global  aerosol  observability  and  the  development  of  realistic  uncertainties  for  composition 
observations,  retrievals  and  model  products.  Outstanding  problems  facing  the  community  relate  to 
such  issues  as  observation  bias,  representativeness,  and  information  spreading  for  the  myriad  of 
sporadic  data  sources  available.  To  meet  this  need,  in  this  grant  we  investigated  the  use  of  these 
diverse  flows  of  data  using  ensemble  and  ensemble  data  assimilation  technologies  to  be  incorporated 
into  the  Navy  Aerosol  Analysis  and  Prediction  System  (NAAPS)/Navy  Variational  Analysis  Data 
Assimilation-Aerosol  Optical  Depth  (NAVDAS-AOD)  framework.  This  work  also  led  to  the 
development  of  the  world’s  first  quasi-operational  global  multi-model  ensemble,  drawing  from  the 
world’s  operational  data  centers.  This  is  the  nucleus  of  a  potential  operational  multi-model  ensemble. 
As  with  other  consensus-like  products,  such  an  ensemble  is  likely  to  remain  top  ranked. 

OBJECTIVES 

Our  overarching  goal  is  to  investigate  applied  science  aerosol  observability  issues  related  to  the  proper 
determination  of  observed  product  efficacy  and  information  spreading.  To  this  end  the  core 
methodology  involves  the  development  and  application  of  an  ensemble  version  of  NAAPS  and  its 
subsequent  ensemble  based  data  assimilation  system.  At  the  same  time,  we  have  initiated  the 
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development  of  a  multi-model  ensemble  across  major  centers  including  European  ECMWF,  Japanese 
JMA,  NASA  GMAO,  NOAA  NCEP  and  the  Spanish  BSC.  Subcomponents  of  our  effort  include: 

1 )  The  development  of  an  ensemble -based  NAAPS  system  which  makes  use  of  the  FNMOC 
NOGAPS  ensemble  data  set:  Here  we  will  use  the  base  NAAPS  model,  including  the  data 
assimilation  system  and  run  in  parallel  with  the  20  NOGAPS  fields,  further  perturbed  with  10 
to  30  additional  perturbations  to  source  and  sink  functions.  This  version  of  NAAPS  will  be 
prepared  for  eventual  transition  to  6.4  and  FNMOC  if  the  Navy  so  desires. 

2)  The  development  of  key  verification  metrics  of  models  and  observations  alike:  Based  on  initial 
simulation  of  the  ensemble,  we  will  investigate  appropriate  error  metrics  for  Navy  satellite  and 
model  products  that  account  for  the  non-Gaussian,  spatially  correlated  nature  of  biases  present 
in  the  environment.  This  objective  includes  the  development  of  probabilistic  tools  for  forecasts. 
These  can  then  be  applied  to  satellite  observations  for  model  verification. 

3)  Investigation  into  the  nature  of  available  satellite  and  in  situ  observations  including  their  flow 
dependant  correlation  lengths  for  use  in  NAVDAS-AOD:  The  optimal  mix  of  meteorological 
and  source/sink  perturbations  for  the  NAAPS  ensemble  for  given  computational  constraints 
will  be  identified.  A  particular  emphasis  will  be  on  the  application  of  these  methods  to  aerosol 
vertical  distribution. 

4)  The  incorporation  of  NAAPS  in  the  NCAR  Data  Assimilation  Test-bed  (DART):  Once  the 
NAAPS  ensemble  model  is  running  quasi-operationally  and  is  well  understood,  we  will  port 
NAAPS  to  the  DART  package  at  NCAR.  This  is  the  first  step  in  developing  our  ensemble- 
based  data  assimilation  system. 

5)  The  creation  of  a  research  grade  ensemble-based  data  assimilation  system:  Based  on  our 
experience  with  the  DART  system,  we  anticipate  being  able  to  have  the  ability  to  harvest  and 
develop  a  robust  ensemble  based  data  assimilation  system.  This  will  be  used  to  supplement  the 
existing  3DVAR  aerosol  data  assimilation  system  currently  in  use.  Emphasis  in  this  system 
will  be  on  sparse  or  isolated  observations  such  as  provided  by  lidars,  sun  photometers  and 
surface  reports.  In  addition,  ensemble  co-variability  studies  will  be  carried  out  to  address 
phenomenological  questions  such  as  modes  of  atmosphere/aerosol  coupling. 

6)  Collaborate  with  other  centers  to  derive  an  aerosol  consensus  for  severe  events:  It  has  been 
shown  that  for  many  extreme  events,  consensus  forecasting  is  a  top  perfonner.  In  this  objective 
we  wish  to  investigate  the  applicability  of  performing  multi-model  consensus  of  GMAO, 
ECMWF  and  other  centers  to  investigate  the  predictability  and  extent  of  significant  aerosol 
events. 

APPROACH 

As  outlined  in  previous  reports,  in  FY12  and  13,  we  developed  the  basic  architecture  of  both  the 
NAAPS  single  and  ICAP -multi-model  ensembles.  Work  perfonned  in  FY13  focuses  on  the  application 
of  these  systems  for  better  understanding  aerosol  observability.  The  spatial  covariance  fields  are 
generated  in  an  ensemble  version  of  NAAPS  (henceforth  eNAAPS).  Nominally  eNAAPS  is  run  quasi- 
operationally  using  the  20  NOGAPS  ensemble  members  generated  by  FNMOC  every  12  hours. 
Additionally,  other  members,  based  on  perturbations  to  the  NAAPS  source  function,  are  created. 


2 


Results  from  the  initial  stages  of  analysis  led  to  a  series  of  upgrades  for  eNAAPS  sources  and 
microphysics  (such  as  a  new  pollution  specie  to  replace  sulfate)  which  are  available  for  the 
detenninistic  operational  version  once  the  NAVGEM  porting  is  complete. 

Collinear  with  the  development  of  eNAAPS  has  been  the  continued  development  of  the  multi-model 
ensemble.  Based  on  members  of  the  International  Cooperative  for  Aerosol  Prediction  (ICAP),  the 
ICAP  Multi-model  ensemble  (or  ICAP-MME)  is  the  world’s  first  quasi-operational  aerosol  ensemble, 
and  is  already  seeing  significant  use  across  centers.  There  are  four  multi  species  models  is  ICAP- 
MME:  ECMWF-MACC,  JMA-MASINGAR,  NASA  GMAO-GEOS-5,  and  a  locally  run  NRL  NAAPS 
next  generation.  In  addition  to  these,  for  the  dust  species  only,  we  also  incorporate  NOAA  NCEP’s 
NGAC  and  the  Barcelona  Supercomputing  dust  models.  Data  for  all  models  now  goes  back  18 
months,  with  the  time  series  for  the  core  models  going  back  28  months. 

Tools  and  display  systems  were  also  developed  for  analysis  of  ensemble  products.  Basic  tools  for 
analyzing  ensemble  variably,  correlation  length  scales,  and  the  most  commonly  employed  “spaghetti 
plots”  and  rank  histograms  were  largely  adapted  from  existing  Matlab  code  into  Perl,  Python  and  IDL 
framework  currently  used  in  7544  and  FNMOC.  Online  verification  products  based  on  AERONET, 
satellite  data  and  own  analyses  are  being  developed.  In  a  similar  manner  to  eNAAPS,  we  cooperate 
with  other  meteorological  centers  over  the  globe  which  are  developing  global  aerosol  prediction 
capabilities.  By  pulling  data  from  such  centers  as  ECMWF,  JMA,  NASA  GMAO,  and  NOAA  NCEP, 
we  created  a  first  of  its  kind  global  aerosol  multi-model  ensemble.  For  comparison  purposes,  products 
generated  are  similar  in  nature  to  eNAAPS. 

For  ensemble  based  data  assimilation  we  ported  NAAPS  into  the  NCAR  Data  Assimilation  Research 
Testbed  (DART)  in  years  one  and  two.  DART  hosts  numerous  ensemble-based  data  assimilation  tools 
including  the  core  components  of  an  EnKF  system.  Both  NOGAPS  and  COAMPS  have  already  been 
ported  to  DART,  and  ensemble  based  COAMPS  dust  source  functions  studies  are  already  underway 
through  a  joint  project  with  Dr.  Hansen.  Hence,  in  house  knowledge  already  exists  to  expedite  this 
process.  Included  in  this  budget  is  additional  travel  money  so  that  NRL  developers  can  spend  the 
required  time  at  NCAR  to  make  this  port  happen. 

Finally,  additional  verification  tools  need  to  be  generated  to  evaluate  improvements  to  the  model.  This 
not  only  allows  model  parameterizations,  but  also  the  satellite  data  assimilated  into  the  models  to  be 
investigated  and  improved.  In  particular,  technologies  need  to  be  created  that  can  quickly  evaluate 
new  or  updated  satellite  products,  such  as  those  being  generated  by  VIIRS. 

WORK  COMPLETED 

In  FY  13,  the  following  tasks  were  completed: 

1)  eNAAPS  was  fully  exercised.  Most  importantly,  in  conjunction  with  Dean  Hegg  (University  of 
Washington)  we  ported  the  current  NAAPS  sulfate  model  to  a  more  generic  “pollution”  specie 
to  account  for  the  more  dominant  secondary  organic  aerosol  loadings  in  the  atmosphere.  Once 
NAVGEM  development  and  tuning  is  complete,  we  feel  the  NAAPS  ensemble  is  ready  for 
transition  to  6.4. 

2)  We  performed  an  extensive  analysis  of  the  ICAP  multi-model  ensemble  for  a  one  year  period 
of  perfonnance.  We  rigorously  compared  ICAP-MEE  and  its  single  model  components  against 
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a  series  of  continuous  (e.g.,  RMSE,  MAE  etc.)  and  threat  score  based  metrics.  Regions  of 
common  success  and  difficulty  over  the  globe  were  mapped. 

3)  Publications  on  eNAAPS  and  ICAP-MME  were  initiated,  and  at  the  end  of  the  fiscal  year 
rough  drafts  of  findings  will  be  complete 

RESULTS 

Research  results  for  this  project  can  be  divided  into  areas  of  ensemble  development,  verification,  and 
data  assimilation.  In  FY1 1  &  12,  focus  was  on  development  and  data  assimilation.  For  FY13,  focus  has 
been  on  verification  and  analysis,  with  additional  effort  on  the  creation  of  a  pollution  specie  in 
NAAPS,  and  the  tuning  of  the  dust  and  smoke  parameters. 

Pollution  development  and  tuning:  Figure  1  demonstrates  before  and  optimized  versions  of  eNAAPS, 
based  on  NOGAPS  deterministic  meteorology.  The  largest  single  improvement  was  due  to  the 
incorporation  of  secondary  organic  aerosol  mass  into  the  sulfate  specie,  creating  a  “fine  mode  pollution 
specie.”  Areas  of  consistent  difficulty  for  NAAPS  including  East  Asia,  the  Indian  sub-continent,  and 
the  eastern  United  States,  have  been  dramatically  improved.  The  mechanism  uses  maps  of  isoprene, 
terpene  and  aromatic  emissions,  largely  from  the  MACC  and  MEGAN  source  functions,  to  develop  a 
new  source  for  fine  mode  particles.  This  source  is  not  static,  but  is  based  on  environmental  conditions 
and  the  pre-existing  aerosol  load.  Thus,  this  formulation  does  not  technically  add  any  additional 
aerosol  or  gas  species  to  be  carried  in  the  model.  In  addition  to  this,  based  on  the  previous  ensemble 
Kalman  filter  work,  the  dust  source  map  and  biomass  burning  emission  have  been  improved.  In  Asia, 
the  Indian  sub-continent  and  sub-Saharan  Africa,  RMSE  values  were  improved  by  40%.  Mean 
innovations  to  account  for  the  lacking  source  function  for  fine  mode  organic  aerosol  particles  were  also 
dramatically  reduced. 

Ensemble  development  and  analysis:  In  year  three  we  examined  a  number  of  case  studies  of  severe 
dust  and  biomass  burning  events.  An  example  of  this  is  depicted  in  Figure  2,  where  both  the  eNAAPS 
“spaghetti  plot”  and  several  other  ICAP-MME  members  are  presented.  In  all  of  these  cases,  while  a 
dust  front  was  predicted  by  all  model  members,  magnitudes  differed  greatly.  Diversity  in  the  models 
could  be  tracked  back  to  a  dust  source  in  the  Taklimakan  desert.  Aerosol  data  assimilation  in  and  of 
itself  does  not  help  in  this  situation,  as  the  dust  is  generated  in  a  desert  area  where  aerosol  optical 
thickness  retrievals  are  rare  and  inaccurate.  Further,  during  transport,  dust  was  masked  by  a  frontal 
cirrus  shield.  However,  using  the  NRL  dust  enhancement  product  we  could  determine  semi- 
quantitatively  which  ensemble  members  were  performing  correctly.  In  an  operational  setting, 
forecasters  could  use  the  ensemble  in  this  way  with  Navy  imagery  to  accurately  forecast  a  wave  of  dust 
passing  over  the  Korean  Peninsula  and  the  Sea  of  Japan. 

Verification:  Both  eNAAPS  and  ICAP  MME  underwent  extensive  verification  and  analysis  in  FY13. 
From  this,  a  common  climatology  was  generated  and  areas  of  common  difficulty  among  models  were 
identified.  Examples  of  both  these  aspects  are  presented  in  Figure  3.  In  the  upper  set  of  panels,  total, 
fine,  and  coarse  mode  AOT  medians  and  geometric  standard  deviations  are  presented  for  the  ICAP- 
MME  ensemble  average.  Inset  as  dots  on  this  figure  are  AERONET  verification  values.  Overall,  we 
find  that  all  models  from  all  centers  tend  to  underestimate  AOT  globally  against  AERONET.  Further, 
while  all  models  reproduce  major  aerosol  features  (e.g.,  dust  plumes  in  Africa  and  Asia,  major  biomass 
burning  regions,  etc.),  there  are  at  times  very  large  differences  in  mean  amplitude  and  minimum  AOT. 
Particularly  large  differences  exist  in  in  the  mid-latitude  oceans,  presumably  due  to  differences  in  sea 
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salt  source  functions.  In  both  cases,  these  are  important  findings  that  will  lead  to  future  work  to  further 
improve  aerosols  source/sink  functions  and  data  assimilation. 

IMPACT/APPLICATIONS 

We  expect  much  of  this  work  to  be  of  immediate  use  to  the  warfighter.  Just  as  the  current  NRL 
aerosol  page  is  frequently  used  in  the  METOC  community,  we  expect  these  ensemble  products  to  be  of 
immediate  applicability.  To  begin  with,  in  key  portions  of  the  globe  we  can  generate  dust,  smoke,  or 
pollution  “Spaghetti  plots”  for  each  of  the  meteorological  members  (Figure  4)  or  independent  models. 
These  can  also  be  shown  spatially  as  areas  of  high  variability  (Figure  2).  In  these  areas  we  will 
collaborate  with  Dr.  Hansen  and  Dr.  Whitcomb  who  are  working  on  METOC  impacts,  decision  aids, 
and  scorecards. 

TRANSITIONS 

We  have  begun  discussions  with  FNMOC  to  transition  eNAAPS  to  operations.  We  are  currently 
waiting  for  the  next  set  of  revisions  of  NAVGEM  to  take  place.  Once  the  NOGAPS  ensemble 
transitions  to  the  NAVGEM  ensemble,  we  anticipate  a  final  round  of  model  optimization  to  take  place 
before  we  initiate  6.4  work. 

RELATED  PROJECTS 

This  project  is  tightly  coupled  to  a  number  of  ONR  32  programs,  particularly  those  of  Professor 
Jianglong  Zhang  at  the  University  of  North  Dakota.  Our  primary  transition  partner  is  Douglas 
Westphal,  who  is  principal  investigator  on  the  Large-Scale  Aerosol  Model  Development  (PI:  Doug 
Westphal).  New  data-processing  and  visualization  systems  are  being  adapted  for  aerosol  research 
through  the  COAMPS-On  Scene  (COAMPS-OSCr)  1  IVPS  charts  program  (PI:  John  Cook).  We  have 
also  begun  working  with  Jim  Hansen  on  his  ONR-funded  project  for  the  use  of  ensemble  data 
assimilation  in  the  prediction  of  atmospheric  constituents. 
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Figure  1.  Original  and  optimized  Aerosol  Optical  Thickness  (AOT)  and  mean  AOT  innovation  with 
the  original  NAAPS,  and  an  optimized  version  which  included  the  incorporation  of  secondary 

organic  aerosol  in  the  model. 
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Figure  2.  An  example  of  differences  within  eNAAPS  and  the  ICAP  Multi-Model  Ensemble  (ICAP- 
MME)  for  a  single  72  hour  dust  event  in  November  2012. 
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Figure  3.  Bi-seasonal  sets  of  statistics  for  the  ICAP  Multi-model  ensemble  for  a  1  year  test  period. 
Upper  set:  the  seasonal  ICAP-MME  median  and  geometric  standard  deviation  of  total,  fine  and 
coarse  mode  AOT.  Lower  set:  a  point-wise  comparison  of  the  maximum  and  minimum  mean  aerosol 
optical  thickness  (AOT)  drawn  from  the  ensemble  members.  That  is,  each  point  on  the  map  draws 
from  the  average  maximum  and  minimum  from  the  ensemble  members,  thus  showing  the  maximum 
average  or  minimum  average  AOT  for  any  point.  Shown  are  total,  fine,  and  coarse  mode  AOT. 

Inset  are  averages  derived  from  AERO  NET 
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